Projection system controlling method, and projector

ABSTRACT

A projection system controlling method includes identifying a first correspondence, identifying a second correspondence, identifying a third correspondence between a plurality of pixels corresponding to an overlap region in an image projected by a first projector, accuracy of the third correspondence is higher than accuracy of the first correspondence, identifying a fourth correspondence between a plurality of pixels corresponding to the overlap region in an image projected by a second projector, accuracy of the fourth correspondence is higher than accuracy of the second correspondence, projecting an image onto a region different from the overlap region based on the first correspondence, projecting an image onto the overlap region based on the third correspondence, projecting an image onto a region different from the overlap region based on the second correspondence, and projecting an image onto the overlap region based on the fourth correspondence.

The present application is based on, and claims priority from JPApplication Serial Number 2021-081094, filed May 12, 2021, thedisclosure of which is hereby incorporated by reference herein in itsentirety.

BACKGROUND 1. Technical Field

The present disclosure relates to a projection system controllingmethod, and a projector.

2. Related Art

There is a known multi-projection system of related art including aplurality of projectors. In the multi-projection system, a tiled imageformed of images projected from the plurality of projectors is displayedon a screen. On the screen, in an overlap region where the region wherethe image projected from one of the plurality of projectors is displayedand the region where the image projected from another projector isdisplayed overlap with each other, the image projected from the oneprojector and the image projected from the other projector overlap witheach other.

In the multi-projection system, the images displayed on the screen areeach distorted due to the performance of the projection lens of theprojector and the angle between the optical axis of the projector andthe screen. The plurality of projectors each therefore performcalibration for correction of the distortion. The calibration includesacquisition of the correspondence between the coordinate system of aprojection apparatus incorporated in the projector and the coordinatesystem of an imaging apparatus incorporated in the projector. Theexecution of the calibration allows each of the plurality of projectorsto correct an input image based on the correspondence and project acorrected image on the screen. For example, in JP-A-2018-152660, asingle calibration image showing a plurality of grid points is used inthe calibration. On the other hand, in JP-A-2007-510966, a plurality ofGray code images different from one another are used as the calibrationimage, and the plurality of Gray code images are displayed on the screenin a time division manner.

Since the images projected from the plurality of projectors overlap witheach other in the overlap regions of the tiled image, it is necessary toimprove the accuracy of the correction of the input images. When aplurality of Gray code images are employed in the calibration performedby each of the projectors, it takes a long time for the calibration ascompared with a case where the calibration is performed by using asingle calibration image.

The present disclosure has been made in view of the circumstancesdescribed above, and an advantage of the present disclosure is reductionin the time required for the calibration with an increase in theaccuracy of the calibration in the overlap region.

SUMMARY

To achieve the advantage described above, a projection systemcontrolling method according to an aspect of the present disclosure is amethod for controlling a projection system which includes a firstprojector including a first imaging apparatus and a second projectorincluding a second imaging apparatus and in which an image projectedfrom the first projector and an image projected from the secondprojector overlap with each other in an overlap region of a displaysurface. The method includes identifying a first correspondence betweena plurality of pixels that form an image projected from the firstprojector onto a first projection region of the display surface and aplurality of pixels that form a first captured image produced by thefirst imaging apparatus through capture of an image of the displaysurface, identifying a second correspondence between a plurality ofpixels that form an image projected from the second projector onto asecond projection region of the display surface and a plurality ofpixels that forma second captured image produced by the second imagingapparatus through capture of an image of the display surface,identifying a third correspondence between a plurality of pixelscorresponding to the overlap region in the image projected by the firstprojector and a plurality of pixels corresponding to the overlap regionin the first captured image, identifying a fourth correspondence betweena plurality of pixels corresponding to the overlap region in the imageprojected by the second projector and a plurality of pixelscorresponding to the overlap region in the second captured image,projecting an image onto a region of the first projection region, theregion being different from the overlap region, based on the firstcorrespondence, projecting an image onto the overlap region based on thethird correspondence, projecting an image onto a region of the secondprojection region, the region being different from the overlap region,based on the second correspondence, and projecting an image onto theoverlap region based on the fourth correspondence, and accuracy of thethird correspondence is higher than accuracy of the firstcorrespondence, and accuracy of the fourth correspondence is higher thanaccuracy of the second correspondence.

To achieve the advantage described above, a projection systemcontrolling method according to another aspect of the present disclosureis a method for controlling a projection system which includes a firstprojector, a second projector, and an imaging apparatus and in which animage projected from the first projector and an image projected from thesecond projector overlap with each other in an overlap region of adisplay surface. The method includes identifying a first correspondencebetween a plurality of pixels that form an image projected from thefirst projector onto a first projection region of the display surfaceand a plurality of pixels that form a captured image produced by theimaging apparatus through capture of an image of the display surface onwhich the image is projected, identifying a second correspondencebetween a plurality of pixels that form an image projected from thesecond projector onto a second projection region of the display surfaceand a plurality of pixels that form a captured image produced by theimaging apparatus through capture of an image of the display surface onwhich the image is projected, identifying a third correspondence betweena plurality of pixels corresponding to the overlap region in the imageprojected by the first projector and a plurality of pixels correspondingto the overlap region in the captured image captured by the imagingapparatus, identifying a fourth correspondence between a plurality ofpixels corresponding to the overlap region in the image projected by thesecond projector and a plurality of pixels corresponding to the overlapregion in the captured image captured by the imaging apparatus,projecting an image onto a region of the first projection region, theregion being different from the overlap region, based on the firstcorrespondence, projecting an image onto the overlap region based on thethird correspondence, projecting an image onto a region of the secondprojection region, the region being different from the overlap region,based on the second correspondence, and projecting an image onto theoverlap region based on the fourth correspondence, and accuracy of thethird correspondence is higher than accuracy of the firstcorrespondence, and accuracy of the fourth correspondence is higher thanaccuracy of the second correspondence.

To achieve the advantage described above, a projection system accordingto an aspect of the present disclosure is a projection system whichincludes a first projector including a first imaging apparatus and asecond projector including a second imaging apparatus and in which animage projected from the first projector and an image projected from thesecond projector overlap with each other in an overlap region of adisplay surface. The first projector identifies a first correspondencebetween a plurality of pixels that form an image projected from thefirst projector onto a first projection region of the display surfaceand a plurality of pixels that form a first captured image produced bythe first imaging apparatus through capture of an image of the displaysurface, identifies a third correspondence between a plurality of pixelscorresponding to the overlap region in the image projected by the firstprojector and a plurality of pixels corresponding to the overlap regionin the first captured image, and projects an image onto a region of thefirst projection region, the region being different from the overlapregion, based on the first correspondence and projects an image onto theoverlap region based on the third correspondence. The second projectoridentifies a second correspondence between a plurality of pixels thatform an image projected from the second projector onto a secondprojection region of the display surface and a plurality of pixels thatform a second captured image produced by the second imaging apparatusthrough capture of an image of the display surface, identifies a fourthcorrespondence between a plurality of pixels corresponding to theoverlap region in the image projected by the second projector and aplurality of pixels corresponding to the overlap region in the secondcaptured image, and projects an image onto a region of the secondprojection region, the region being different from the overlap region,based on the second correspondence and projects an image onto theoverlap region based on the fourth correspondence. Accuracy of the thirdcorrespondence is higher than accuracy of the first correspondence, andaccuracy of the fourth correspondence is higher than accuracy of thesecond correspondence.

To achieve the advantage described above, a projection system accordingto another aspect of the present disclosure is a projection system whichincludes a first projector, a second projector, and an imaging apparatusand in which an image projected from the first projector and an imageprojected from the second projector overlap with each other in anoverlap region of a display surface. The first projector identifies afirst correspondence between a plurality of pixels that form an imageprojected from the first projector onto a first projection region of thedisplay surface and a plurality of pixels that form a first capturedimage produced by the imaging apparatus through capture of an image ofthe display surface, identifies a third correspondence between aplurality of pixels corresponding to the overlap region in the imageprojected by the first projector and a plurality of pixels correspondingto the overlap region in the first captured image, and projects an imageonto a region of the first projection region, the region being differentfrom the overlap region, based on the first correspondence and projectsan image onto the overlap region based on the third correspondence. Thesecond projector identifies a second correspondence between a pluralityof pixels that form an image projected from the second projector onto asecond projection region of the display surface and a plurality ofpixels that form a second captured image produced by the imagingapparatus through capture of an image of the display surface,identifying a fourth correspondence between a plurality of pixelscorresponding to the overlap region in the image projected by the secondprojector and a plurality of pixels corresponding to the overlap regionin the second captured image, and projects an image onto a region of thesecond projection region, the region being different from the overlapregion, based on the second correspondence and projects an image ontothe overlap region based on the fourth correspondence. Accuracy of thethird correspondence is higher than accuracy of the firstcorrespondence, and accuracy of the fourth correspondence is higher thanaccuracy of the second correspondence.

To achieve the advantage described above, a projector according to anaspect of the present disclosure is a projector including an imagingapparatus, a projection apparatus, and a processing apparatus. Theprocessing apparatus identifies a first relationship that is acorrespondence between a plurality of pixels that form an imageprojected from the projection apparatus onto a projection region of adisplay surface and a plurality of pixels that form a captured imageproduced by the imaging apparatus through capture of an image of thedisplay surface, and identifies a second relationship that is acorrespondence between a plurality of pixels corresponding to an overlapregion of an image projected by the projection apparatus, the overlapregion being a region where the image projected by the projectionapparatus and an image projected from a projector different from theprojector overlap with each other, and a plurality of pixelscorresponding to the overlap region in the captured image. Theprojection apparatus projects an image onto a region of the projectionregion, the region being different from the overlap region, based on thefirst relationship and projects an image onto the overlap region basedon the second relationship. Accuracy of the second relationship ishigher than accuracy of the first relationship.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an example of the configuration of a projection systemaccording to an embodiment.

FIG. 2 shows a screen on which a tiled image is displayed.

FIG. 3 shows an example of the configuration of a projector.

FIG. 4 is a flowchart showing an example of the action of the projector.

FIG. 5 is a flowchart showing an example of calibration performed by theprojector.

FIG. 6 diagrammatically shows the projector that performs a calibrationmode.

FIG. 7 diagrammatically shows the projector that performs thecalibration mode.

FIG. 8 diagrammatically shows the screen on which Gray code images aredisplayed.

FIGS. 9A and 9B show examples of Gray code images displayed in anoverlap region.

FIG. 10 is a flowchart showing one step according to a variation indetail in the action of the projector.

FIG. 11 shows an example of the configuration of a projection systemaccording to a variation.

DESCRIPTION OF EXEMPLARY EMBODIMENTS 1. Embodiment

A preferable embodiment of the present disclosure will be describedbelow with reference to the drawings. In the drawings, the dimensionsand scale of each portion differ from the actual values as appropriate.The drawings are diagrammatically drawn in some cases so as to be easilyunderstood. Furthermore, the scope of the present disclosure is notlimited to the forms presented below by way of example unless otherwisestated to specifically limit the present disclosure.

FIG. 1 shows an example of the configuration of a projection system 100according to the present embodiment. The projection system 100 includesa projector 10A and a projector 10B. The present embodiment will bedescribed with reference to the case where the two projectors 10A and10B are provided by way of example. In the following description, whenit is not particularly necessary to distinguish the projector 10A andthe projector 10B from each other, they are called the projectors 10.The projector 10A is an example of a “first projector”, and theprojector 10B is an example of a “second projector”.

The projection system 100 includes an image processing apparatus 20, animage supplier 30, an operation apparatus 40, and a screen SC, as shownin FIG. 1 . The projector 10A, the projector 10B, and the imageprocessing apparatus 20 are communicably connected to each other. Theconnection among the projector 10A, the projector 10B, and the imageprocessing apparatus 20 may be wireless or wired connection, orconnection with a network, such as the Internet or a LAN (local areanetwork), interposed among them. In the present embodiment, theprojector 10A functions as a master projector, and the projector 10Bfunctions as a slave projector.

It is assumed in FIG. 1 to use axes X, Y, and Z perpendicular to oneanother. The direction along the axis X viewed from an arbitrary pointis referred to as an axis-X direction. Similarly, the direction alongthe axis Y viewed from an arbitrary point is referred to as an axis-Ydirection, and the direction along the axis Z viewed from an arbitrarypoint is referred to as an axis-Z direction. A plane X-Y including theaxes X and Y corresponds to a horizontal plane. The axis Z is an axialline along the vertical direction. In the present embodiment, the screenSC is disposed so as to stand vertically. The projection system 100 caninstead be so configured that the screen SC is disposed so as to beparallel to the horizontal plane, and that the projector 10 is disposedabove the screen SC in the vertical direction.

The projectors 10A and 10B project images G1 and G2 on the screen SC,respectively. The images G1 and G2 projected on the screen SC form atiled image TG. The front surface of the screen SC functions as adisplay surface S, where the tiled image TG is displayed. In thefollowing description, when it is not necessary to distinguish theimages G1 and G2 from each other, they are referred to as images G.

The display surface S has a first projection region 2A and a secondprojection region 2B. The first projection region 2A and the secondprojection region 2B form a projection region 2 of the display surfaceS. The first projection region 2A corresponds to the projector 10A andis a region where the image G1 is displayed. The second projectionregion 2B corresponds to the projector 10B and is a region where theimage G2 is displayed. The first projection region 2A and the secondprojection region 2B overlap with each other at the display surface S toform an overlap region 3. The first projection region 2A is formed ofthe overlap region 3 and a non-overlap region 21A. The non-overlapregion 21A is a region as a result of removal of the overlap region 3from the first projection region 2A. The second projection region 2B isformed of the overlap region 3 and a non-overlap region 21B. Thenon-overlap region 21B is a region as a result of removal of the overlapregion 3 from the second projection region 2B.

The image processing apparatus 20 is coupled to each of the projector10A and the projector 10B, as shown in FIG. 1 . The image processingapparatus 20 divides image data TD acquired from the image supplier 30,which will be described later, to generate input image data D1 and theinput image data D2. The image processing apparatus 20 outputs the inputimage data D1 to the projector 10A and outputs the input image data D2to the projector 10B. In the present embodiment, the projector 10Aprojects the image G1 onto the first projection region 2A based on theinput image data D1, and the projector 10B projects the image G2 ontothe second projection region 2B based on the input image data D2, sothat the tiled image TG is displayed on the display surface S. In thefollowing description, when it is not necessary to distinguish the inputimage data D1 and the input image data D2 from each other, they arereferred to as input image data D.

FIG. 2 shows an example of the configuration of the tiled image TG, withthe screen SC, on which the tiled image TG is displayed, viewed in theaxis-Y direction. In FIG. 2 , the displayed images G1 and G2 are shiftedfrom each other in the axis-Z direction for convenience of description,but the images G1 and G2 are actually aligned with each other at thesame height position. Furthermore, the first projection region 2A islarger than the image G1, but the first projection region 2A and theimage G1 actually have the same size, and the circumferential edge ofthe first projection region 2A coincides with the circumferential edgeof the image G1. Similarly, the second projection region 2B is largerthan the image G2, but the second projection region 2B and the image G2actually have the same size, and the circumferential edge of the secondprojection region 2B coincides with the circumferential edge of theimage G2.

The tiled image TG is formed of the images G1 and G2 arranged side byside in the axis-X direction. In the present embodiment, the projector10 are disposed in the plane X-Y, and the images G1 and G2 arranged inthe axis-X direction form the tiled image TG. The tiled image TG in thepresent embodiment, however, does not necessarily have the configurationshown in FIG. 2 and may be formed of images projected from theprojectors 10 stacked in the axis-Z direction.

The image processing apparatus 20 is an information processingapparatus, such as a PC (personal computer), a smartphone, or a tabletterminal each including a processor that is not shown. The imageprocessing apparatus 20 is coupled to the image supplier 30, as shown inFIG. 1 . The image supplier 30 supplies the image processing apparatus20 with the image data TD corresponding to the tiled image TG. The imagedata TD may be data based on still images or may be data based on motionimages. The image supplier 30 is realized by a medium reproductionapparatus, such as a video reproduction apparatus or a DVD (digitalversatile disc) apparatus, or an information processing apparatus, suchas a PC, a smartphone, and a tablet terminal. The image processingapparatus 20 may have part or entirety of the functions of the imagesupplier 30.

The image processing apparatus 20 is coupled to the operation apparatus40, as shown in FIG. 1 . The operation apparatus 40 accepts anoperator's input relating to the setting of a display mode of the tiledimage TG. The operation apparatus 40 instead accepts the operator'sinput relating to the setting of the size of the tiled image TG and theresolution thereof. The operation apparatus 40 further accepts theoperator's input relating to the setting of the size of the overlapregion 3 and the position of the overlap region 3 on the display surfaceS. The operation apparatus 40 may be integrated with the imageprocessing apparatus 20.

FIG. 3 shows an example of the configuration of each of the projectors10. The projectors 10 each include a processing apparatus 210, a storageapparatus 220, a communication apparatus 230, an input apparatus 240, animaging apparatus 250, a projection apparatus 260, and an imageprocessing circuit 270, as shown in FIG. 3 .

The processing apparatus 210 is an apparatus that controls each portionof the projector 10. The processing apparatus 210 includes a processor,such as a CPU (central processing unit). The processing apparatus 210may be formed of a single processor or a plurality of processors. Partor entirety of the functions of the processing apparatus 210 may berealized by hardware, such as a DSP (digital signal processor), an ASIC(application specific integrated circuit), a PLD (programmable logicdevice), and an FPGA (field programmable gate array).

The storage apparatus 220 stores a program PG executed by the processingapparatus 210, and a lookup table LUT, calibration image data Dc, andmonochromatic image data Dx used by the processing apparatus 210. Thecalibration image data Dc represents a calibration image having aplurality of grid points. The monochromatic image data Dx represents amonochromatic image. The lookup table LUT will be described later indetail. The storage apparatus 220 includes, for example, a hard diskdrive or a semiconductor memory. Part or entirety of the storageapparatus 220 may be provided in a storage apparatus or any othercomponent external to the projector 10.

The communication apparatus 230 is a communication circuit communicablycoupled to the image processing apparatus 20. The communicationapparatus 230 includes an interface, for example, a USB (universalserial bus) and a LAN. The communication apparatus 230 acquires theinput image data D outputted from the image processing apparatus 20. Theinput image data D represents input images. The communication apparatus230 may be wirelessly connected to the image processing apparatus 20,for example, in accordance with Wi-Fi or Bluetooth, or may be connectedto the image processing apparatus 20 via, for example, a LAN or theInternet. Wi-Fi and Bluetooth are each a registered trademark.

The input apparatus 240 is, for example, an operation panel including avariety of switches and provided as part of an enclosure, which is notshown, of the projector 10. Specifically, the input apparatus 240 isformed, for example, of a power switch that powers on and off theprojector 10, a switch that causes the projector 10 to start projectingan image, and a switch that invokes a menu via which the projector 10 isset.

The imaging apparatus 250 generates captured image data representing acaptured image by capturing an image of the display surface S of thescreen SC. The imaging apparatus 250 includes an imaging device 251, asshown in FIG. 3 . The imaging apparatus 250 is, for example, a camera.

The imaging device 251 is, for example, an image sensor such as a CCD(charge coupled device) or a CMOS (complementary MOS). The imagingdevice 251 captures an image of the display surface S and outputscaptured image data to the processing apparatus 210.

The projection apparatus 260 projects the image G onto the displaysurface S of the screen SC. The projection apparatus 260 includes alight source 261, an optical modulator 262, and a projection opticalsystem 263, as shown in FIG. 3 .

The light source 261 is formed, for example, of a halogen lamp, a xenonlamp, an ultrahigh-pressure mercury lamp, an LED (light emitting diode),or a laser light source. The optical modulator 262 modulates the lightemitted by the light source 261 to generate image light. The opticalmodulator 262 includes, for example, three transmissive or reflectiveliquid crystal panels corresponding to the three primary colors of lightof R, G, and B light. The optical modulator 262 may instead beconfigured to include a light modulation device, such as a digitalmirror device, and a color wheel. The projection optical system 263guides the image light modulated by the optical modulator 262 to thescreen SC and brings the image light into focus on the display surfaceS. The projection optical system 263 may further include a zoommechanism that enlarges or reduces the image G to be displayed on thedisplay surface S of the screen SC and a focus adjustment mechanism thatperforms focus adjustment.

The image processing circuit 270 performs predetermined image processingon the input image data D acquired by the communication apparatus 230.Examples of the image processing performed by the image processingcircuit 270 include not only keystone correction but also digitalzooming, color tone correction, and luminance correction.

The image G projected from each of the projectors 10 and displayed onthe display surface S is distorted in accordance with the angle betweenthe optical axis of the projector 10 and the display surface S. Thekeystone correction corrects trapezoidal distortion produced on thedisplay surface S when the image G is projected from the projector 10.The keystone correction corrects the input image based on thecorrespondence between a camera coordinate system and a projectorcoordinate system. The camera coordinate system is a coordinate systemof a captured image captured by the imaging apparatus 250. The projectorcoordinate system is a coordinate system of the image G projected fromthe projector 10. The correspondence between the camera coordinatesystem and the projector coordinate system represents the correspondencebetween certain coordinates in the captured image and the correspondingcoordinates in the image G. In other words, the correspondence betweenthe camera coordinate system and the projector coordinate system is thecorrespondence between a plurality of pixels that form the capturedimage and a plurality of pixels that form the image G. The lookup tableLUT stores the correspondence between the plurality of pixels that formthe captured image and the plurality of pixels that form the image G.

The image processing circuit 270 refers to the lookup table LUT andperforms the keystone correction on the input image data D to generatecorrected image data Dh. The image processing circuit 270 outputs thecorrected image data Dh to the optical modulator 262, so that the imageG based on the corrected image data Dh is displayed on the screen SC.

The processing apparatus 210 reads the program PG from the storageapparatus 220 and executes the read program PG to control the entireprojector 10. The processing apparatus 210 performs first calibrationand second calibration before projecting the image G based on the inputimage data Don the display surface S. The two-step calibrationidentifies the correspondence between the plurality of pixels that formthe captured image and the plurality of pixels that form the image G.

FIG. 4 is a flowchart showing an example of the action of the projector10A. The action of the projector 10A will be described below withreference to FIG. 4 as appropriate. In the following description, asubscript a is added to an element relating to the projector 10A, and asubscript b is added to an element relating to the projector 10B.

A processing apparatus 210 a first performs the first calibration instep St1. FIG. 5 is a flowchart showing the first calibration in detail.Step St1 will be described in detail with reference to FIG. 5 asappropriate.

When the processing apparatus 210 a accepts an instruction of start ofthe calibration, the processing apparatus 210 a reads in step St11calibration image data Dca from a storage apparatus 220 a and outputsthe read data to a projection apparatus 260 a. The projection apparatus260 a projects a first calibration image CG1 indicated by thecalibration image data Dca on the display surface S. The firstcalibration image CG1 is thus displayed on the display surface S, asshown in FIG. 6 . The first calibration image CG1 according to thepresent embodiment is a dot pattern image containing a plurality offirst markers CP1, as shown in FIG. 6 . The plurality of first markersCP1 are regularly arranged in a grid pattern. Examples of theaforementioned instruction of start of the calibration may include aninstruction issued when an input apparatus 240 a is operated and aninstruction based on a control program.

An imaging apparatus 250 a then captures an image of the display surfaceS on which the first calibration image CG1 is projected in step St12under the control of the processing apparatus 210 a. The imagingapparatus 250 a is an example of a “first imaging apparatus”. The dotpattern contained in the captured image that is a captured displaysurface S on which the first calibration image CG1 is projected isdistorted in accordance with the position where the imaging apparatus250 a is installed. The processing apparatus 210 a processes each of thefirst markers CP1 in the captured image to identify the coordinates ofthe first marker CP1 in the camera coordinate system. Since the firstmarkers CP1 each have an area, the processing apparatus 210 a calculatesthe center of gravity of each of the first markers CP1 in the capturedimage, and uses the result of the calculation as the coordinates of thefirst marker CP1 in the camera coordinate system. The coordinates in thecamera coordinate system are expressed by the position of each of thepixels that form an imaging device 251 a. On the other hand, the firstmarkers CP1 in the first calibration image CG1 each have knowncoordinates in the projector coordinate system. The coordinates of eachof the first markers CP1 mean the coordinates of the center of gravityof the first marker CP1. The plurality of pixels that form the firstcalibration image CG1 correspond to the pixels of each of the liquidcrystal panels of the optical modulator 262 a. The coordinates in theprojector coordinate system are therefore indicated by the positions ofthe pixels that form the liquid crystal panel.

In step St13, the processing device 210 a subsequently causes theplurality of pixels that form the captured image to correspond to theplurality of pixels that form the image G1 based on the coordinates ofthe first markers CP1 in the captured image and the coordinates of thefirst markers CP1 in the first calibration image CG1. Since the firstmarkers CP1 are regularly arranged in the first calibration image CG1,the correspondence between the plurality of pixels that form the firstcalibration image CG1 and the plurality of pixels that form the capturedimage cannot be instantly identified. The processing apparatus 210 atherefore causes the plurality of pixels that form the captured image tocorrespond to the plurality of pixels that form the image G1 byperforming interpolation. The processing apparatus 210 a thus identifiesa first correspondence between the plurality of pixels that form theimage G1 projected from the projector 10A onto the display surface S andthe plurality of pixels that form the captured image created by theimaging apparatus 250 a through capture of an image of the displaysurface S on which the image G1 is projected. The processing apparatus210 a stores the first correspondence in a lookup table LUTa. Thecaptured image is an example of a “first captured image”, and the firstcorrespondence is an example of a “first relationship”.

The processing apparatus 210 a subsequently causes in step St14 theprojection apparatus 260 a to stop projecting the first calibrationimage CG1. In step St15, the processing apparatus 210 a notifies aprocessing apparatus 210 b that the preceding step St13 has beencompleted. Upon acquisition of the notification, the processingapparatus 210 b performs the same first calibration performed in thepreceding steps St11 to St13, as the processing apparatus 210 a does.

FIG. 7 shows a second calibration image CG2 projected on the displaysurface S. In step St11, a projection apparatus 260 b projects thesecond calibration image CG2 onto the display surface S, as theprojection device 260 a does. The second calibration image CG2 is thusdisplayed on the display surface S, as shown in FIG. 7 . The secondcalibration image CG2 according to the present embodiment is a dotpattern image containing a plurality of second markers CP2, as shown inFIG. 7 . The plurality of second markers CP2 are regularly arranged in agrid pattern.

An imaging apparatus 250 b captures an image of the display surface S onwhich the second calibration image CG2 is projected in step St12. Theimaging apparatus 250 b is an example of a “second imaging apparatus”.

The processing apparatus 210 b thus identifies in step St13 a secondcorrespondence between the plurality of pixels that form the image G2projected from the projector 10B onto the second projection region 2Band the plurality of pixels that form the captured image created by theimaging apparatus 250 b through capture of an image of the displaysurface S on which the image G2 is projected. The processing apparatus210 b stores the second correspondence in a lookup table LUTb.

Returning to FIG. 4 , the processing apparatus 210 a then identifies theoverlap region 3 in step St2. Specifically, the processing apparatus 210a reads monochromatic image data Dxa from the storage apparatus 220 aand outputs the monochromatic image data Dxa to the projection apparatus260 a. The monochromatic image data Dxa represents a monochromatic imageas described above. The projection apparatus 260 a projects themonochromatic image onto the first projection region 2A in accordancewith the data. The monochromatic image is an example of a “firstprojection image”.

The processing apparatus 210 a transmits a projection request thatinstructs the projector 10B to project a monochromatic image. Uponreceipt of the projection request, the processing apparatus 210 b of theprojector 10B reads monochromatic image data Dxb from the storageapparatus 220 b and outputs the monochromatic image data Dxb to theprojection apparatus 260 b. The projection apparatus 260 b projects themonochromatic image onto the second projection region 2B in accordancewith the data. The monochromatic image is an example of a “secondprojection image”.

The monochromatic image projected from the projection apparatus 260 aand the monochromatic image projected from the projection apparatus 260b overlap with each other on the display surface S. Now, themonochromatic image projected from the projection apparatus 260 aoverlaps with the monochromatic image projected from the projectionapparatus 260 b on the display surface S. Therefore, out of themonochromatic image projected from the projection apparatus 260 a, theregion that overlaps with the monochromatic image projected from theprojection apparatus 260 b is brighter than the region different fromthe overlap region. Similarly, the monochromatic image projected fromthe projection apparatus 260 b overlaps with the monochromatic imageprojected from the projection apparatus 260 a on the display surface S.Therefore, out of the monochromatic image projected from the projectionapparatus 260 b, the region that overlaps with the monochromatic imageprojected from the projection apparatus 260 a is brighter than theregion different from the overlap region. The monochromatic imagesprojected from the projection apparatuses 260 a and 260 b are eachtypically a white image, but not necessarily, and may instead be amonochromatic image having a color different from white.

The imaging apparatus 250 a subsequently captures an image of the firstprojection region 2A that displays the monochromatic image and outputscaptured image data representing the captured image under the control ofthe processing apparatus 210 a. The processing apparatus 210 a thencompares the luminance of each of the plurality of pixels that form thecaptured image with a threshold based on the captured image data. Basedon the result of the comparison, the processing apparatus 210 aidentifies a region containing pixels having luminance greater than orequal to the threshold as the overlap region 3. The threshold is set soas to allow determination of the overlap region 3. Specifically, let Rbe the threshold, X be the luminance of the monochromatic imageprojected from the projection apparatus 260 a, and Y be the luminance ofthe monochromatic image projected from the projection apparatus 260 b,and the threshold R may satisfy X<R<X+Y. It is preferable that R=(2X+Y)/2 in consideration of a margin.

Carrying out the processes described above allows identification of theoverlap region 3 in the camera coordinate system. The processingapparatus 210 a identifies the overlap region 3 in the projectorcoordinate system based on the first correspondence generated by thefirst calibration in the preceding step St1.

The processing apparatus 210 a then transmits an imaging request thatinstructs the projector 10B to capture an image of the second projectionregion 2B that displays the monochromatic image. The processingapparatus 210 b of the projector 10B having received the imaging requestproduces a captured image by causing the imaging apparatus 250 b tocapture an image of the second projection region 2B that displays themonochromatic image. The processing apparatus 210 b compares theluminance of each of the plurality of pixels that form the capturedimage with a threshold based on the captured image data representing thecaptured image. Based on the result of the comparison, the processingapparatus 210 b identifies a region containing pixels having luminancegreater than or equal to the threshold as the overlap region 3. Thethreshold is set so as to allow determination of the overlap region 3.Specifically, let r be the threshold, x be the luminance of themonochromatic image projected from the projection apparatus 260 b, and ybe the luminance of the monochromatic image projected from theprojection apparatus 260 a, and the threshold r may satisfy x<r<x+y. Itis preferable that r=(2 x+y)/2 in consideration of a margin.

Carrying out the processes described above allows identification of theoverlap region 3 in the camera coordinate system. The processingapparatus 210 b identifies the overlap region 3 in the projectorcoordinate system based on the first correspondence generated by thefirst calibration in the preceding step St1.

The processing apparatus 210 a generates a plurality of Gray code imagesG3 different from one another in step St3. The Gray code images G3 arean example of “first space code images”. The processing apparatus 210 adetermines the number of Gray code images G3 in accordance with thehorizontal and vertical resolution of the overlap region 3. Thehorizontal resolution of the overlap region 3 is the number of pixels ofthe image G1 corresponding to the overlap region 3, the pixels beingthose arranged in the axis-X direction. The vertical resolution of theoverlap region 3 is the number of pixels of the image G1 correspondingto the overlap region 3, the pixels being those arranged in the axis-Zdirection. The shape of the overlap region 3 changes in accordance withthe state in which the projectors 10A and 10B are installed. Theprocessing apparatus 210 a generates a plurality of Gray code images G3different from one another in accordance with the overlap region 3identified in step St2. The processing apparatus 210 a therefore doesnot need to generate the Gray code image G3 more than necessary.

The projection apparatus 260 a then projects in step St4 the pluralityof Gray code images G3 different from each other onto the displaysurface S in a time-division manner. FIG. 8 diagrammatically shows thescreen SC on which the Gray code images G3 are displayed.

Under the control of the processing apparatus 210 a, the imagingapparatus 250 a produces captured images by capturing images of theoverlap region 3 that displays the Gray code images G3. The capturedimages are an example of “third captured images”. The imaging apparatus250 a captures an image of the overlap region 3 whenever the pluralityof Gray code images G3 different from one another are each displayed inthe overlap region 3.

The processing apparatus 210 a identifies in step St5, based on aplurality of captured images corresponding in a one-to-onecorrespondence to the plurality of Gray code images G3 and produced bythe imaging apparatus 250 a through capture of images of the overlapregion 3 where the plurality of Gray code images G3 are projected, thirdcorrespondence between a plurality of pixels corresponding to theoverlap region 3 in the image G1 and a plurality of pixels correspondingto the overlap region 3 in the captured images captured by the imagingapparatus 250 a. Specifically, the processing apparatus 210 a identifiesthe third correspondence by analyzing a binary code, which will bedescribed later, indicated by each of the pixels of the image from theprojection apparatus 260 a that correspond to the pixels of the overlapregion 3. The processing apparatus 210 a updates a portion of the firstcorrespondence stored in the lookup table LUTa, the portioncorresponding to the overlap region 3 of the image G1, to the thirdcorrespondence. The lookup table LUTa thus stores the firstcorrespondence for the non-overlap region 21A and the thirdcorrespondence for the overlap region 3. The third correspondence is anexample of a “second relationship”.

The relationship between the plurality of pixels that form the image G1and a Gray code pattern will now be described. FIGS. 9A and 9B showexamples of the Gray code images G3 displayed in the overlap region 3.FIG. 9A shows Gray code images G3 each formed of black and white stripeshaving a longitudinal direction extending along the axis-Z direction.FIG. 9B shows Gray code images G3 each formed of black and white stripeshaving a longitudinal direction extending along the axis-X direction.The Gray code images G3 are each a kind of image expressed by bright anddark portions and representing a binary code having black/white binaryvalues or illuminated and non-illuminated states, and is skillfullyconfigured to be robust against noise. In the present embodiment, theprojection apparatus 260 a displays the Gray code images G3 in FIGS. 9Aand 9B sequentially from left to right in the figure in the overlapregion 3. In this process, assigning different time-variant luminancepatterns formed of black having a luminance of 0 and white having aluminance of 1 to all the pixels of an image from the projectionapparatus 260 a causes the pixels of the image from the projectionapparatus 260 a to correspond to binary codes different from oneanother. That is, the pixels of the image from the projection apparatus260 a are each caused to correspond to a binary code and emit light in ameasurement pattern representing the unique binary code. Therefore, whenthe Gray code images G3 shown in FIG. 9A are formed N Gray code imagesG3, a unique binary code is given to each of 2^(N) pixels in the axis-Xdirection out of the pixels of the image from the projection apparatus260 a that correspond to the overlap region 3. Similarly, when the Graycode images G3 shown in FIG. 9B are formed M Gray code images G3, aunique binary code is given to each of 2^(M) pixels in the axis-Zdirection out of the pixels of the image from the projection apparatus260 a that correspond to the overlap region 3. That is, using the N Graycode images G3 shown in FIG. 9A and the M Gray code images G3 shown inFIG. 9B allows a unique binary code to be given to each of the2^(N)×2^(M) pixels of the image from the projection apparatus 260 a thatcorrespond to the overlap region 3. The imaging apparatus 250 atherefore records the number of the illuminated one of the Gray codeimages G3 and the number of the non-illuminated one of the Gray codeimages G3 to allow identification of the correspondence between themeasurement pattern observed at each of the pixels of the image from theimaging apparatus 250 a and the binary code at a pixel of the image fromthe projection apparatus 260 a. The number of necessary binary codes istherefore determined in accordance with the number of pairs of a pixelin the camera coordinate system and a pixel in the projector coordinatesystem, the pixels desired to acquire the correspondence, in the axis-Xand axis-Z directions. For example, the smaller the horizontalresolution of the overlap region 3, the smaller the number of necessarybinary codes.

The process in step St3 to the process in step St5 described abovecorrespond to the second calibration, which is performed in the overlapregion 3 and causes the camera coordinate system to correspond to theprojector coordinate system. In the first calibration, the calibrationis performed by using a single first calibration image CG1. In the firstcalibration image CG1, a plurality of first markers CP1, which serve asa positioning reference, are arranged. The first calibration canidentify the correspondence between the pixel of the image G1 at which afirst marker CP1 is located and a pixel among the plurality of pixelsthat form the captured image. The correspondence between the pixelslocated between first markers CP1 adjacent to each other and pixels ofthe captured image is, however, identified by the interpolation. On theother hand, in the second calibration, since the third correspondence isidentified by using a plurality of Gray code images G3, the accuracy ofthe correspondence in the overlap region 3 is higher than the accuracyof the correspondence in the non-overlap region 21A. Therefore,according to the present embodiment, the accuracy of the calibration inthe overlap region 3 can be increased as compared with the accuracy ofthe calibration in the non-overlap region 21A.

The processing apparatus 210 a subsequently transmits in step St6completion notification indicating that the second calibration has beencompleted to the projector 10B via the communication apparatus 230 a.Upon receipt of the completion notification, the projector 10B startsthe second calibration, as the projector 10A does. Specifically, theprocessing apparatus 210 b carries out the process in step St3 to theprocess in step St5.

The processing apparatus 210 b generates a plurality of Gray code imagesG3 different from one another in step St3. The Gray code images G3 arean example of “second space code images”.

The projection apparatus 260 b projects in step St4 each of theplurality of Gray code images G3 different from each other onto thedisplay surface S in a time division manner. Under the control of theprocessing apparatus 210 b, the imaging apparatus 250 b then producescaptured images by capturing images of the overlap region 3 thatdisplays the Gray code images G3. The captured images are an example of“fourth captured images”. The imaging apparatus 250 b captures an imageof the overlap region 3 whenever the plurality of Gray code images G3different from one another are each displayed in the overlap region 3.

The processing apparatus 210 b identifies in step St5, based on aplurality of captured images corresponding in a one-to-onecorrespondence to the plurality of Gray code images G3 and produced bythe imaging apparatus 250 b through capture of images of the overlapregion 3 where the plurality of Gray code images G3 are projected,fourth correspondence between a plurality of pixels corresponding to theoverlap region 3 in the image G2 and a plurality of pixels correspondingto the overlap region 3 in the captured images captured by the imagingapparatus 250 b. Specifically, the processing apparatus 210 b identifiesthe fourth correspondence by analyzing the binary code indicated by eachof the pixels of the image from the projection apparatus 260 b thatcorrespond to the pixels of the overlap region 3. The processingapparatus 210 b updates a portion of the second correspondence stored inthe lookup table LUTb, the portion corresponding to the overlap region 3of the image G2, to the fourth correspondence. The lookup table LUTbthus stores the second correspondence for the non-overlap region 21B andthe fourth correspondence for the overlap region 3. The processingapparatus 210 b then transmits completion notification indicating thatthe second calibration has been completed to the projector 10A via thecommunication apparatus 230 b.

In step St7, the processing apparatus 210 a evaluates whether or not thecompletion notification has been received from the projector 10B, andrepeats the evaluation until the result of the evaluation becomesaffirmative. When the completion notification is received from theprojector 10B, the result of the evaluation becomes affirmative, and theprocessing apparatus 210 a projects the image G1 onto the displaysurface S in step St8. In step St8, the processing apparatus 210 aoutputs the input image data D1 to the image processing circuit 270 a.The image processing circuit 270 a refers to the lookup table LUTa toperform the keystone correction on the input image data D1. The imageprocessing circuit 270 a outputs the corrected image data Dh produced bythe keystone correction to the optical modulator 262 a, so that theimage G1 based on the corrected image data Dh is displayed on the screenSC. The lookup table LUTa stores the first correspondence for thenon-overlap region 21A and the third correspondence for the overlapregion 3. Therefore, according to the present embodiment, the accuracyof the keystone correction in the overlap region 3 can be higher thanthe accuracy of the keystone correction in the non-overlap region 21A.

After the second calibration is completed, the projector 10B projectsthe image G2 onto the display surface S as in step St8. Specifically,the processing apparatus 210 b outputs the input image data D2 to theimage processing circuit 270 b. The image processing circuit 270 brefers to the lookup table LUTb to perform the keystone correction onthe input image data D2. The image processing circuit 270 b outputs thecorrected image data Dh produced by the keystone correction to theoptical modulator 262 b, so that the image G2 based on the correctedimage data Dh is displayed on the screen SC.

As understood from the above description, the projection system 100provided with the projector 10A including the imaging apparatus 250 aand the projector 10B including the imaging apparatus 250 b executes acontrol method for displaying the image G1 projected from the projector10A and the image G2 projected from the projector 10B with the images G1and G2 overlapping with each other in the overlap region 3 of thedisplay surface S. Specifically, the projection system 100 executes acontrol method including steps St1, St5, and St8 described above. Theprojection system 100 identifies in step St1 the first correspondencebetween the plurality of pixels that form the image G1 projected fromthe projector 10A onto the first projection region 2A of the displaysurface S and the plurality of pixels that form the first captured imageproduced by the imaging apparatus 250 a through capture of an image ofthe display surface S, as described above. The projection system 100further identifies in step St1 the second correspondence between theplurality of pixels that form the image G2 projected from the projector10B onto the second projection region 2B of the display surface S andthe plurality of pixels that form a second captured image produced bythe imaging apparatus 250 b through capture of an image of the displaysurface S. The projection system 100 identifies in step St5 the thirdcorrespondence between the plurality of pixels corresponding to theoverlap region 3 in the image G1 projected by the projector 10A and theplurality of pixels corresponding to the overlap region 3 in the firstcaptured image. The projection system 100 further identifies in step St5the fourth correspondence between the plurality of pixels correspondingto the overlap region 3 in the image G2 projected by the projector 10Band the plurality of pixels corresponding to the overlap region 3 in thesecond captured image. The projection system 100 projects in step St8,based on the first correspondence, the image G1 onto the non-overlapregion 21A, which is a region different from the overlap region 3 out ofthe first projection region 2A, and the image G1 onto the overlap region3 based on the third correspondence, which is more accurate than thefirst correspondence. The projection system 100 further projects in stepSt8, based on the second correspondence, the image G2 onto thenon-overlap region 21B, which is a region different from the overlapregion 3 out of the second projection region 2B, and the image G2 ontothe overlap region 3 based on the fourth correspondence, which is moreaccurate than the second correspondence.

The present disclosure, in which the accuracy of calibration in theoverlap region 3 can be higher than the accuracy of calibration in theother regions or the non-overlap regions 21A and 21B as described above,allows improvement in the quality of the image in the overlap region 3,where the image G1 projected from the projector 10A and the image G2projected from the projector 10B overlap with each other, compared to acase where inaccurate calibration is performed across the region.Furthermore, since the highly accurate calibration only needs to beperformed in the overlap region 3, the time required for the calibrationis shorter than in a case where the highly accurate calibration isperformed across the region. Therefore, according to the presentdisclosure, the time necessary for the calibration can be shortened withthe accuracy of the calibration in the overlap region 3 improved.

The projector 10A identifies the third correspondence based on aplurality of captured images produced by projecting a plurality of Graycode images G3 different from each other onto the overlap region 3 in atime-division manner and causing the imaging apparatus 250 a to captureimages of the overlap region 3 on which the plurality of Gray codeimages G3 are projected and corresponding in one-to-one correspondenceto the plurality of Gray code images G3, and the projector 10Bidentifies the fourth correspondence based on a plurality of capturedimages produced by projecting a plurality of Gray code images G3different from each other onto the overlap region 3 in a time-divisionmanner and causing the imaging apparatus 250 b to capture images of theoverlap region 3 on which the plurality of Gray code images G3 areprojected and corresponding in one-to-one correspondence to theplurality of Gray code images G3.

According to the present disclosure, since the Gray code images G3 areprojected only onto the overlap region 3, the number of Gray code imagesG3 necessary for the identification of the third and fourthcorrespondence only needs to be the number according to the resolutionof the overlap region 3. Excessive time will therefore not be spent toidentify the third and fourth correspondence.

The number of plurality of Gray code images G3 to be generated by theprojector 10A changes in accordance with the number of a plurality ofpixels corresponding to the overlap region 3 in an image projected bythe projector 10A, and the number of plurality of Gray code images G3 tobe generated by the projector 10B changes in accordance with the numberof a plurality of pixels corresponding to the overlap region 3 in animage projected by the projector 10B. The projectors 10A and 10Btherefore do not need to generate the Gray code images G3 more thannecessary.

The first correspondence is identified based on the first calibrationimage CG1 projected from the projector 10A onto the first projectionregion 2A and a captured image produced by the imaging apparatus 250 athrough capture of an image of the display surface S on which the firstcalibration image CG1 is projected. The second correspondence isidentified based on the second calibration image CG2 projected from theprojector 10B onto the second projection region 2B and a captured imageproduced by the imaging apparatus 250 b through capture of an image ofthe display surface S on which the second calibration image CG2 isprojected. The first calibration image CG1 contains the plurality ofregularly arranged first markers CP1. The second calibration image CG2contains the plurality of regularly arranged second markers CP2.

The projector 10A identifies as the overlap region 3 the regioncontaining pixels having luminance greater than a threshold out of aplurality of pixels that form a captured image produced by the imagingapparatus 250 a through capture of an image of the first projectionregion 2A on which a monochromatic image is projected, the projector 10Bidentifies as the overlap region 3 the region containing pixels havingluminance greater than a threshold out of a plurality of pixels thatform a captured image produced by the imaging apparatus 250 b throughcapture of an image of the second projection region 2B on which amonochromatic image is projected. The projectors 10A and 10B canselectively project the Gray code images G3 only onto the overlap region3 of the display surface S by identifying the overlap region 3.

2. Variations

The aforementioned forms presented by way of example can be changed in avariety of manners. Aspects of specific variations applicable to each ofthe aforementioned forms are presented below byway of example. Two ormore of the aspects arbitrarily selected from those presented below byway of example may be combined with each other as appropriate to theextent that the selected aspects do not contradict each other.

Variation 1

In the aspect described above, the projection system 100 is configuredto include the two projectors 10, but not necessarily, and may beconfigured to include two or more projectors 10.

Variation 2

In the aspect described above, the Gray code images G3 are projectedonto the overlap region 3, but not necessarily, and space code imagesdifferent from the Gray code images G3 may be projected. Examples of thespace code images may include phase shift images.

Variation 3

The method for identifying the overlap region 3 is not limited to themethod presented by way of example in the aspect described above. Theprojector 10 may identify the overlap region 3, for example, by using amethod shown in FIG. 10 . FIG. 10 is a flowchart showing step St2according to Variation 3 in detail. Step St2 will be described belowwith reference to FIG. 10 as appropriate.

First, in step St21, the processing apparatus 210 a reads themonochromatic image data Dxa stored in advance in the storage apparatus220 a and outputs the data to the projection apparatus 260 a. Theprojection apparatus 260 a projects the monochromatic image onto thedisplay surface S in accordance with the data. The contour of themonochromatic image coincides with the contour of the first projectionregion 2A. Thereafter, in step St22, the imaging apparatus 250 acaptures an image of the display surface S that displays themonochromatic image projected from the projection apparatus 260 a underthe control of the processing apparatus 210 a. Subsequently, in stepSt23, the processing apparatus 210 a identifies the contour of the firstprojection region 2A from the captured image that is the capturedmonochromatic image. Specifically, in step St23, the processingapparatus 210 a identifies the pixels corresponding to the contour ofthe monochromatic image in the camera coordinate system out of thepixels of the captured image that is the captured display surface S thatdisplays the monochromatic image projected from the projection apparatus260 a. The processing apparatus 210 a then refers to the lookup tableLUTa, which stores the first correspondence, to identify the pixelscorresponding to the contour of the monochromatic image in the projectorcoordinate system of the projector 10A.

Thereafter, in step St24, the imaging apparatus 250 b captures an imageof the display surface S that displays the monochromatic image projectedfrom the projection apparatus 260 a, as the imaging apparatus 250 adoes. Subsequently, in step St25, the processing apparatus 210 bidentifies the contour of the first projection region 2A from thecaptured image that is the captured monochromatic image. Specifically,in step St25, the processing apparatus 210 b identifies the pixelscorresponding to the contour of the monochromatic image in the cameracoordinate system out of the pixels of the captured image that is thecaptured display surface S that displays the monochromatic imageprojected from the projection apparatus 260 a. The processing apparatus210 b then refers to the lookup table LUTb, which stores the secondcorrespondence, to identify the pixels corresponding to the contour ofthe monochromatic image in the projector coordinate system of theprojector 10B. The processing apparatus 210 a then causes the projectionapparatus 260 a to stop projecting the monochromatic image.

Subsequently, in step St26, the processing apparatus 210 b reads themonochromatic image data Dxb stored in advance in the storage apparatus220 b and outputs the data to the projection apparatus 260 b. Theprojection apparatus 260 b projects the monochromatic image onto thedisplay surface S in accordance with the data. The contour of themonochromatic image coincides with the contour of the second projectionregion 2B. Thereafter, in step St27, the imaging apparatus 250 acaptures an image of the display surface S that displays themonochromatic image projected from the projection apparatus 260 b underthe control of the processing apparatus 210 a. Subsequently, in stepSt28, the processing apparatus 210 a identifies the contour of thesecond projection region 2B from the captured image, which is thecaptured monochromatic image, as in the preceding step St23.

Thereafter, in step St29, the imaging apparatus 250 b captures an imageof the display surface S that displays the monochromatic image projectedfrom the projection apparatus 260 b, as the imaging apparatus 250 adoes. Subsequently, in step St30, the processing apparatus 210 bidentifies the contour of the second projection region 2B from thecaptured image that is the captured monochromatic image, as in thepreceding step St25.

Subsequently, in step St31, the processing apparatus 210 a identifies asthe overlap region 3 in the projector coordinate system the regionsurrounded by the contour of the first projection region 2A identifiedin the preceding step St23 and the contour of the second projectionregion 2B identified in the preceding step St28. Thereafter, in stepSt32, the processing apparatus 210 b identifies as the overlap region 3in the projector coordinate system the region surrounded by the contourof the first projection region 2A identified in the preceding step St25and the contour of the second projection region 2B identified in thepreceding step St30, as the processing apparatus 210 a does.

As is understood from the above description, the projection system 100according to Variation 3 executes the control method described above.The control method to be executed includes steps St23, St25, St28, St30,St31, and St32 described above. As described above, in step St23, theprojector 10A identifies the contour of the first projection region 2Abased on the captured image produced by the imaging apparatus 250 athrough capture of an image of the display surface S on which themonochromatic image is projected. In step St25, the projector 10Bidentifies the contour of the first projection region 2A based on thecaptured image produced by the imaging apparatus 250 b through captureof an image of the display surface S on which the monochromatic image isprojected. In step St28, the projector 10A identifies the contour of thesecond projection region 2B based on the captured image produced by theimaging apparatus 250 a through capture of an image of the displaysurface S on which the monochromatic image is projected. In step St30,the projector 10B identifies the contour of the second projection region2B based on the captured image produced by the imaging apparatus 250 bthrough capture of an image of the display surface S on which themonochromatic image is projected. In steps St31 and St32, the projectors10A and 10B identify as the overlap region 3 the region surrounded bythe identified contour of the first projection region 2A and theidentified contour of the second projection region 2B, respectively.When the projection system 100 according to Variation 3 executes thecontrol method described above, the overlap region 3 is identified bythe projectors 10A and 10B. The projectors 10A and 10B can thereforeselectively project the Gray code images G3 only onto the overlap region3 of the display surface S.

Variation 4

FIG. 11 shows an example of the configuration of a projection system100A according to Variation 4. In the aspect described above, theprojectors 10 are each configured to include the imaging apparatus 250,but not necessarily, and the projection system 100A may be configured toinclude an imaging apparatus 50 separate from the projectors 10. Theimaging apparatus 50 is communicably coupled to each of the projectors10 or the image processing apparatus 20, has the same configuration asthat of the imaging apparatus 250 in the aspect described above, andoperates in the same manner in accordance with which the imagingapparatus 250 operates. The projectors 10 according to the Variation 4each have the same configuration as that of the projectors 10 in theaspect described above except that the imaging apparatus 250 is removedfrom each of the projectors 10 in the aspect described above. Theprojection system 100A according to the Variation 4 operates in the samemanner in accordance with which the projection system 100 in the aspectdescribed above operates.

As is understood from the above description, the projection system 100Aprovided with the projector 10A, the projector 10B, and the imagingapparatus 50 executes a control method for displaying the image G1projected from the projector 10A and the image G2 projected from theprojector 10B with the images G1 and G2 overlapping with each other inthe overlap region 3 of the display surface S. Specifically, theprojection system 100A executes a control method including steps St1,St5, and St7 described above. In step St1, the projection system 100Aidentifies the first correspondence between the plurality of pixels thatform the image G1 projected from the projector 10A onto the firstprojection region 2A of the display surface S and the plurality ofpixels that form the captured image produced by the imaging apparatus 50through capture of an image of the display surface S on which the imageG1 is projected, as described above. In step St1, the projection system100A further identifies the second correspondence between the pluralityof pixels that form the image G2 projected from the projector 10B ontothe second projection region 2B of the display surface S and theplurality of pixels that form the captured image produced by the imagingapparatus 50 through capture of an image of the display surface S onwhich the image G2 is projected. In step St5, the projection system 100Aidentifies the third correspondence between the plurality of pixelscorresponding to the overlap region 3 in the image G1 projected by theprojector 10A and the plurality of pixels corresponding to the overlapregion 3 in the image captured by the imaging apparatus 50. In step St5,the projection system 100A further identifies the fourth correspondencebetween the plurality of pixels corresponding to the overlap region 3 inthe image G2 projected by the projector 10B and the plurality of pixelscorresponding to the overlap region 3 in the image captured by theimaging apparatus 50. In step St7, the projection system 100A projects,based on the first correspondence, the image G1 onto the non-overlapregion 21A, which is the region different from the overlap region 3, ofthe first projection region 2A, and the image G1 onto the overlap region3 based on the third correspondence, which is more accurate than thefirst correspondence. In step St7, the projection system 100A furtherprojects, based on the second correspondence, the image G2 onto thenon-overlap region 21B, which is the region different from the overlapregion 3, of the second projection region 2B, and the image G2 onto theoverlap region 3 based on the fourth correspondence, which is moreaccurate than the second correspondence. The control method provides thesame effects and advantages as those provided by the method forcontrolling the projection system 100 according to the aspect describedabove.

Variation 5

In the aspects described above, the processing apparatus 210 identifiesthe overlap region 3 from a captured image that is a captured displaysurface S that displays a monochromatic image, but not necessarily. Theprocessing apparatus 210 may identify the overlap region 3 based, forexample, on data inputted from the operator via the operating apparatus40 and representing the widths of the overlap region 3 in the axis-X andthe axis-Z directions.

Variation 6

In the aspects described above, the projection apparatus 260 may projectthe Gray code images G3 different from one another in a time divisionmanner onto an enlarged overlap region encompassing the overlap region 3identified in step St2 in the aspects described above. The Gray codeimages G3 can therefore be displayed in the overlap region 3 even whenthe accuracy of the identification of the overlap region 3 is low instep St2 in the aspects described above. The enlarged overlap region isformed of a plurality of pixels corresponding to the overlap region 3identified in step St2 in the aspects described above out of the pixelsthat form the liquid crystal panels of the optical modulator 262, and aplurality of pixels present around the plurality of pixels.

Variation 7

In the above aspects described above, the lookup table LUT is used toperform the keystone correction, but not necessarily in the presentdisclosure. For example, the keystone correction may be performed byusing a projection transformation matrix that causes a plurality ofpixels that form the image G projected from each of the projectors 10 tocorrespond to a plurality of pixels that forma captured image.

Variation 8

In the aspects described above, the first calibration and the secondcalibration are performed by the processing apparatus 210 provided inthe projector 10, but not necessarily in the present disclosure. Forexample, the first calibration and the second calibration may beperformed under the control of the image processing apparatus 20 coupledto the projector 10 so as to be communicable with respect to each other.In this case, the first calibration image CG1 and the Gray code imagesG3 may be generated by the image processing apparatus 20 and transmittedto the projector 10A. Similarly, the second calibration image CG2 andthe Gray code images G3 may be generated by the image processingapparatus 20 and transmitted to the projector 10B. Furthermore, an imagecaptured by the imaging apparatus 50 or 250 may be received to identifythe correspondence between the imaging apparatus used to receive thecaptured image and the projector 10. Moreover, data representing theidentified correspondence may be transmitted to the projectors 10, or animage for projection generated based on the identified correspondencemay be transmitted to the projectors 10.

What is claimed is:
 1. A projection system controlling methodcomprising: identifying a first correspondence between a plurality ofpixels of an image projected from a first projector onto a firstprojection region of a display surface and a plurality of pixels of afirst captured image obtained by capturing the display surface with afirst camera provided in the first projector; identifying a secondcorrespondence between a plurality of pixels of an image projected froma second projector onto a second projection region of the displaysurface and a plurality of pixels of a second captured image obtained bycapturing the display surface with a second camera provided in thesecond projector; identifying a third correspondence having an accuracyhigher than an accuracy of the first correspondence between a pluralityof pixels corresponding to an overlap region in the image projected bythe first projector, the overlap region overlapping with the imageprojected from the second projector on the display surface, and aplurality of pixels corresponding to the overlap region in the firstcaptured image; identifying a fourth correspondence having an accuracyhigher than an accuracy of the second correspondence between a pluralityof pixels corresponding to the overlap region in the image projected bythe second projector and a plurality of pixels corresponding to theoverlap region in the second captured image; projecting, by the firstprojector, an image onto a region of the first projection regiondifferent from the overlap region based on the first correspondence;projecting, by the first projector, an image onto the overlap regionbased on the third correspondence; projecting, by the second projector,an image onto a region of the second projection region different fromthe overlap region based on the second correspondence; and projecting,by the second projector, an image onto the overlap region based on thefourth correspondence, wherein the first projector projects a pluralityof first space code images different from one another onto the overlapregion in time division, the third correspondence is identified based oneach of a plurality of third captured images obtained by capturing theoverlap region on which the plurality of first space code images areprojected with the first camera and corresponding in a one-to-onecorrespondence to the plurality of first space code images, the secondprojector projects a plurality of second space code images differentfrom one another onto the overlap region in time division, the fourthcorrespondence is identified based on each of a plurality of fourthcaptured images obtained by capturing the overlap region on which theplurality of second space code images are projected with the secondcamera and corresponding in a one-to-one correspondence to the pluralityof second space code images, a number of the plurality of first spacecode images varies in accordance with a number of a plurality of pixelscorresponding to the overlap region in the image projected by the firstprojector, and a number of the plurality of second space code imagesvaries in accordance with a number of a plurality of pixelscorresponding to the overlap region in the image projected by the secondprojector.
 2. The projection system controlling method according toclaim 1, wherein the first space code images and the second space codeimages are each a Gray code image.
 3. The projection system controllingmethod according to claim 1, wherein the first correspondence isidentified based on a first calibration image projected from the firstprojector onto the first projection region and the first captured imageobtained by capturing the display surface on which the first calibrationimage is projected with the first camera, the second correspondence isidentified based on a second calibration image projected from the secondprojector onto the second projection region and the second capturedimage obtained by capturing the display surface on which the secondcalibration image is projected with the second camera, the firstcalibration image contains a plurality of regularly arranged firstmarkers, and the second calibration image contains a plurality ofregularly arranged second markers.
 4. The projection system controllingmethod according to claim 1, wherein the overlap region is identifiedbased on a captured image obtained by capturing the display surface onwhich a first projection image is projected from the first projector anda second projection image is projected from the second projector.
 5. Theprojection system controlling method according to claim 4, wherein thefirst projector identifies as the overlap region a region containingpixels having luminance greater than or equal to a threshold out of aplurality of pixels of a captured image obtained by capturing the firstprojection region on which the first projection image is projected withthe first camera, and the second projector identifies as the overlapregion a region containing pixels having luminance greater than or equalto the threshold out of a plurality of pixels of a captured imageobtained by capturing the second projection region on which the secondprojection image is projected with the second camera.
 6. The projectionsystem controlling method according to claim 4, wherein the firstprojector identifies a contour of the first projection region based on acaptured image obtained by capturing the display surface on which thefirst projection image is projected with the first camera, andidentifies a contour of the second projection region based on a capturedimage obtained by capturing the display surface on which the secondprojection image is projected with the first camera, the secondprojector identifies the contour of the first projection region based ona captured image obtained by capturing the display surface on which thefirst projection image is projected with the second camera, andidentifies the contour of the second projection region based on acaptured image obtained by capturing the display surface on which thesecond projection image is projected with the second camera, and thefirst projector and the second projector identify a region surrounded bythe contour of the first projection region and the contour of the secondprojection region as the overlap region respectively.
 7. The projectionsystem controlling method according to claim 4, wherein the firstprojection image and the second projection image are each amonochromatic image.
 8. The projection system controlling methodaccording to claim 1, wherein the first projector projects the pluralityof first space code images different from one another in time divisiononto an enlarged overlap region that encompasses the overlap region ofthe display surface, and the second projector projects the plurality ofsecond space code images different from one another onto the enlargedoverlap region in time division.
 9. A projection system controllingmethod comprising: identifying a first correspondence between aplurality of pixels of an image projected from a first projector onto afirst projection region of a display surface and a plurality of pixelsof a first captured image obtained by capturing the display surface onwhich the image is projected from the first projector with a camera;identifying a second correspondence between a plurality of pixels of animage projected from a second projector onto a second projection regionof the display surface and a plurality of pixels of a second capturedimage obtained by capturing the display surface on which the image isprojected from the second projector with the camera; identifying a thirdcorrespondence having an accuracy higher than an accuracy of the firstcorrespondence between a plurality of pixels corresponding to an overlapregion of the image projected by the first projector, the overlap regionoverlapping with the image projected from the second projector on thedisplay surface, and a plurality of pixels corresponding to the overlapregion in the first captured image captured by the camera; identifying afourth correspondence having an accuracy higher than an accuracy of thesecond correspondence between a plurality of pixels corresponding to theoverlap region of the image projected by the second projector and aplurality of pixels corresponding to the overlap region in the secondcaptured image captured by the camera; projecting, by the firstprojector, an image onto a region of the first projection regiondifferent from the overlap region based on the first correspondence;projecting, by the first projector, an image onto the overlap regionbased on the third correspondence; projecting, by the second projector,an image onto a region of the second projection region different fromthe overlap region based on the second correspondence; and projecting,by the second projector, an image onto the overlap region based on thefourth correspondence, wherein the first projector projects a pluralityof first space code images different from one another onto the overlapregion in time division, the third correspondence is identified based oneach of a plurality of third captured images obtained by capturing theoverlap region on which the plurality of first space code images areprojected with the camera and corresponding in a one-to-onecorrespondence to the plurality of first space code images, the secondprojector projects a plurality of second space code images differentfrom one another onto the overlap region in time division, the fourthcorrespondence is identified based on each of a plurality of fourthcaptured images obtained by capturing the overlap region on which theplurality of second space code images are projected with the camera andcorresponding in a one-to-one correspondence to the plurality of secondspace code images, a number of the plurality of first space code imagesvaries in accordance with a number of a plurality of pixelscorresponding to the overlap region in the image projected by the firstprojector, and a number of the plurality of second space code imagesvaries in accordance with a number of a plurality of pixelscorresponding to the overlap region in the image projected by the secondprojector.
 10. A projector comprising: a camera; a projection opticalsystem; and one or more processors programmed to: identify a firstrelationship that is a correspondence between a plurality of pixels ofan image projected via the projection optical system onto a projectionregion of a display surface and a plurality of pixels of a capturedimage obtained by capturing the display surface with the camera,identify a second relationship having an accuracy higher than anaccuracy of the first relationship that is a correspondence between aplurality of pixels corresponding to an overlap region of the imageprojected via the projection optical system, the overlap region being aregion where the image projected via the projection optical system andan image projected from another projector different from the projectorincluding the projection optical system on the display surface overlapwith each other, and a plurality of pixels corresponding to the overlapregion in the captured image, project, via the projection opticalsystem, an image onto a region of the projection region different fromthe overlap region, based on the first relationship and project an imageonto the overlap region based on the second relationship, wherein theprojection optical system projects a plurality of first space codeimages different from one another onto the overlap region in timedivision, the second correspondence is identified based on each of aplurality of second captured images obtained by capturing the overlapregion on which the plurality of first space code images are projectedwith the camera and corresponding in a one-to-one correspondence to theplurality of first space code images, a number of the plurality of firstspace code images varies in accordance with a number of a plurality ofpixels corresponding to the overlap region in the image projected by theprojection optical system, and a number of the plurality of second spacecode images varies in accordance with a number of a plurality of pixelscorresponding to the overlap region in the image projected by the otherprojector.